DOCUMENT RESUME 



ED 452 362 



CE 081 622 



AUTHOR 

TITLE 



PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Wahlstrom, Darryl A. 

The Relationship between Goal Orientation and the 
Calibration of Performance Expectations to Performance 
Feedback . 

2001-04-11 

5 5p . ; Paper presented at the Annual Meeting of the American 
Educational Research Association (Seattle, WA, April 10-14, 
2001 ) . 

Reports - Research (143) -- Speeches/Meeting Papers (150) 

MF01/PC03 Plus Postage. 

Achievement Need; Behavior Development; Cognitive 
Structures; Competence; Educational Research; Expectation; 
Feedback; *Goal Orientation; Labor Force Development; 
Learning Motivation; Multiple Regression Analysis; 
Performance; Secondary Education; *Self Efficacy; *Self 
Evaluation (Individuals); Skills; ^Student Attitudes; 
Student Evaluation; Teacher Response; ^Vocational Education 



ABSTRACT 



A study investigated 210 secondary vocational/ technical 
education students' cognitive, affective, and behavioral responses to a skill 
assessment process . Students received feedback on their skill strengths and 
weaknesses and those skills in which they over- or under-estimated their 
abilities. Results from hierarchical multiple regression indicated certain 
main effects. Students adopting mastery goals held adaptive motivational 
beliefs, including high levels of self-efficacy and feedback seeking 
intentions. Students oriented toward performance-approach goals also held 
higher levels of self-efficacy. Performance-avoidant goals were not 
predictive of students' post-assessment motivation. Structural equation 
models indicated that both calibration accuracy and utility mediate these 
relationships, depending on the skill set investigated. (Appendixes include 4 
figures, 5 tables, and 121 references.) (YLB) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



Goal Orientation and Calibration l 



Running head: GOAL ORIENTATION AND CALIBRATION OF PERFORMANCE 



The Relationship between Goal Orientation and the 
Calibration of Performance Expectations to Performance Feedback 

Darryl A. Wahlstrom 
University of Michigan 



Paper Presented at the American Educational Research Association (AERA) 

National Conference 
Seattle, Washington 
April 10-14, 2001 



Correspondence about this paper can be addressed to the author by e-mail: wahlstro@umich.edu 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

Dr This document has been reproduced as 
/ received from the person or organization 
originating it. 



□ Minor changes have been made to 
improve reproduction quality. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



7 > 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



./ 



BEST COPY AVAILABLE 



Goal Orientation and Calibration 2 



Abstract 

This study investigated two hundred ten (n = 210) secondary vocational/technical education 
students’ cognitive, affective, and behavioral responses to a skill assessment process. Students 
received feedback on their skill strengths and weaknesses and which skills they over- or under- 
estimated their abilities. Results from hierarchical multiple regression indicate certain main 
effects. Students adopting mastery goals held adaptive motivational beliefs including high levels 
of self-efficacy and feedback seeking intentions. Students oriented toward performance-approach 
goals also held high levels of self-efficacy. Performance-avoidant goals were not predictive of 
students’ post-assessment motivation. Structural equation models indicate that both calibration 
accuracy and utility mediate these relationships depending on the skill-set investigated. Results 
from the study are discussed in relation to students’ motives for goal adoption. 
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The Relationship between Goal Orientation and the 
Calibration of Performance Expectations to Performance Feedback 

Workplaces in the 21 st century will feature many changes due to globalization of world 
economies, increased use of technology in the workforce, and increased knowledge and skill 
requirements of workers (Resnick & Wirt, 1996). These resulting high performance workplaces 
will emphasize work teams (Sundstrom, De Meuse, & Futrell, 1990) and work that is configured 
to allow workers more functional authority (Barker, 1993). In light of these impending changes, 
however, reports indicate a lack of qualified candidates for new or vacant positions because 
candidates lack relevant basic, technical, organizational, and company-specific skills and work 
experiences (Bailey, 1991; National Association of Manufacturers [NAM], 1997; O'Neil Jr., 
1997). 

Workforce development programs represent efforts to bolster relationships between 
education and business. The goal of these programs is to ameliorate the impact of an unskilled 
workforce by assessing individuals’ skill levels to identify extant skill gaps and to bridge the 
identified gaps by providing individuals with the competencies necessary for successful 
employment. Such education and training is typically aimed at technical skills, problem-solving 
skills, and “soft” skills such as communication and teamwork (National Center for Research in 
Vocational Education [NCRVE], 1999, March). 

Although skill assessment is an integral component of both employee selection and 
workforce development programs, approaches to assessment need to change to keep pace with 
the changing workforce and workplaces (Herriot & Anderson, 1997). Called for changes include 
assessments that (a) are more efficient, (b) focus on specific, critical job situations, (c) are 
oriented toward ‘meta-criteria’ such as problem solving, interpersonal skills, and 
planning/organizing, (d) emphasize self-assessment of existing skill levels, and (e) assess both 
performance and employee potential (Jansen, 1997). Technology-mediated assessment processes 
have facilitated many of these changes (Frank & Jaffee, 1995). 
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Problem 

Despite the importance of skill assessment, however, little is known about the impact — 
the cognitive, affective, and behavioral responses to feedback — that assessment processes and 
outcomes have on individual participants. This line of research was originally suggested by 
Dreher and Sackett (1983), but it has been relatively unexplored. The few studies that have 
attempted to address this issue have relied on assessment centers as a research context (e.g., lies 
& Robertson, 1989; Robertson, lies, Gratton, & Sharpley, 1991). These same researchers, 
however, point out that methodological weaknesses in their studies clearly delimit the validity 
and generalizability of their research findings, and so only scant evidence of candidates’ 
perceptions of assessment experiences exists. 

In addition, there is relatively little in the way of theoretical ideas that attempt to 
conceptualize the effects of assessment processes and outcomes. Fletcher (1991) claims that the 
paucity of research in this field precludes researchers from making specific hypotheses. Future 
research needs to focus on the extent to which results, procedures, or both, are involved in 
determining candidates’ post-assessment attitudes, intentions, and behavior (Noe, Wilk, Mullen, 
& Wanek, 1997). Future studies also need to determine how responses to feedback may differ 
across individuals (Renn & Prien, 1995). An investigation of feedback effects in concert with 
individual learner motivation and attitudes would be useful in developing a unifying and 
clarifying framework of feedback processes (Mory, 1996) . 

Purpose 

The present study addressed these stated research needs by examining individuals’ 
perceptions of skill assessment feedback as part of a secondary manufacturing technology 
program with vocational and technical education students. The purpose of the study was to 
understand whether a specific motivational variable (e g., goal orientation) was predictive of 
students’ perceptions of the assessment process (fairness, utility, and perceived threat) and their 
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post-assessment motivation (self-efficacy and feedback seeking intent). This study also explored 
whether the goal orientation construct could predict the accuracy of students’ self-assessed 
abilities in relation to their actual assessment performance. 

Conceptual and Theoretical Model 



Overview of Model 

The theoretical model for this study was drawn from the feedback literature (e.g., Fedor, 
1991; Ilgen, Fisher, & Taylor, 1979; Taylor, Fisher, & Ilgen, 1984) and is illustrated 
conceptually in Figure 1. The model suggests that students’ post-assessment motivation is 
affected directly by the types of achievement goals students adopt. In addition, the model depicts 
three potential mediational channels, or routes: 

Insert Figure 1 approx imately here] 

In general terms, the entire model is representative of the processes related to skill 
assessment feedback. This notion is supportive of two concepts in the training literature: Framing 
effects (Quinones, 1995) and training “episodes” (Baldwin & Magjuka, 1997). As such, the 
outcome measures in Figure 1 serve as indices of participants’ motivation for entering 
subsequent “episodes” (i.e., training and education). 

Inline with the views expressed by lies and Robertson (1989), participants’ reactions to 
assessment processes and results serve as cognitive mediators between the independent and 
dependent measures. Similar to extant research on feedback models, attention is given to 
individual difference variables (Renn & Prien, 1995); specifically, the goal orientation construct 
is placed antecedent in the model (c.f., VandeWalle & Cummings, 1997) and is used to predict 
the relationships appearing in the model. 

Placement of the variable “discrepancy score” (an index of students’ calibration 
accuracy) prior to cognitive mediators is suggestive of a causal flow whereby perceptions are 
formed in response to feedback information. This notion is in alignment with several lines of 
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inquiry including control theory (Carver & Scheier, 2000), Feedback Intervention Theory (FIT) 
(Kluger & DeNisi, 1996), and social cognitive theory (Cervone, 1993). 

The outcome measures chosen for this study (self-efficacy and feedback-seeking 
intentions) are inline with contemporary feedback models (Fedor, 1991) and have found 
relevance and import in both educational (Schunk, 1990) and training contexts (Kraiger, Ford, & 
Salas, 1993; Mathieu & Martineau, 1997). Such outcomes, it has been argued, are superior to 
traditional measures (e.g., Kirtpatrick's typology, 1976) because they can be leveraged prior, 
during, and after training events. 

Main (Direct) Effect Relationships 



Feedback-Seeking 

Several research studies have established an understanding of the relationship between 
goal orientation and post-assessment motivation as operationalized in the present study. Two 
studies (VandeWalle, 1997; VandeWalle & Cummings, 1997) found participants’ willingness to 
seek feedback was negatively and significantly related to the performance-avoid goal orientation, 
unrelated to the performance-approach orientation, and positively and significantly related to a 
learning goal orientation. 

The results for the performance-approach orientation, however, have not been 
unequivocal. Using an instrument designed for the academic domain, VandeWalle (1995) found 
a significant negative relationship between feedback seeking intent and performance-approach 
goals (r = 14, p < .05), but in the work domain, the correlation was non-significant (r = -.06, p 

= .33). 

Research on the related construct avoidance of help seeking has also yielded equivocal 
results. Middleton and Midgley (1997) found performance-approach goals to be unrelated to 
avoidance of help seeking (P = .09, n.s.), while Ryan and Pintrich (1997) found a significant, 
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positive relationship albeit it completely mediated via students’ attitudes — namely threat from 
peers and teachers 1 . 

Based on the extant research reviewed, the following empirical research hypothesis is 
proposed: Feedback-seeking intent will be positively related to a mastery goal orientation, 
negatively related to a performance-avoid goal orientation, and unrelated to a performance- 
approach goal orientation (Hypothesis 1). 

Self-Efficacy 

Prior research has investigated the relationship between goal orientation and self- 
efficacy. Middleton and Midgley (1997), for example, found that academic self-efficacy was 
positively and significantly related to task (learning) goals, negatively related to performance- 
avoid goals, but unrelated to performance-approach goals. Results for performance-approach 
goals, though, have been mixed. Anderman and Young (1994) found a significant negative 
correlation between performance-approach goals and perceived academic efficacy while 
Wolters, Yu, and Pintrich (1996) found a positive relationship. Based on the above research, the 
following testable hypothesis is presented: Self-efficacy will be positively related to a mastery 
goal orientation, negatively related to a performance-avoid goal orientation, and unrelated to a 
performance-approach goal orientation (Hypothesis 2). 



1 Strictly speaking, help seeking and avoidance of help seeking are not “opposite” constructs because they represent 
different types of goals. 
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Mediational (Indirect) Effects 

The conceptual and theoretical model proposed for this study argues for three potential 
mediational channels, or routes, for the relationship between students’ goal orientation and their 
post-assessment motivation. Figure 2 represents a “streamlined” version of the full model and 
allows for a clearer understanding of these proposed relationships. 

insert Figure 2 approximately here! 

Mediational Route #1 

Linkages 2 and 3 in Figure 2 depict the path for this proposed mediational route. 
Substantial research in the self-assessment tradition exists to conclude that self-perceptions of 
ability (even if biased) remain valid measures of performance prediction (Assor, Tzelgov, Thein, 
Ilardi, & Connell, 1990; Harter, 1985; Phillips, 1984). What is needed, however, is a theory that 
accounts for the variables that influence self-assessment accuracy and the effect that self- 
assessment accuracy has on one’s subsequent behavior (Heneman, 1980). Achievement goal 
theory (Dweck, 1986; Dweck & Leggett, 1988) may provide such an understanding. 

Contemporary goal achievement theory seeks to understand the reasons why individuals 
adopt particular goal pursuits. Individuals oriented toward mastery goals seek to develop 
competence and to improve skills. Individuals rooted in performance-approach goals seek to 
demonstrate their competence relative to others, whereas individuals with a performance- 
avoidance orientation set the explicit goal of avoiding the demonstration of lack of ability (Elliot 
& Church, 1997; Elliot & Sheldon, 1997; Middleton & Midgley, 1997; Pintrich, 2000a) . 

Brown (1990) relied on tenets of social comparison theory (Festinger, 1954) and 
concluded that even though individuals prefer feedback that reinforces notions of high (rather 
than low) ability, individuals do not totally refrain from seeking feedback that may disclose 
incompetence. There are two motivational influences that operate in ability self-assessments: 
Self-evaluation (to obtain accurate evaluation information) and self- enhancement (to enhance the 
perception of one’s competence). These dual influences can make feedback conflicting because 




9 



Goal Orientation and Calibration 9 



individuals must resolve the tension between acquiring information for its instrumental value and 
also wishing to protect their ego and self-esteem (Northcraft & Ashford, 1990). The findings 
suggest that both self-enhancement and self-evaluation motives affect the dynamics of ability 
evaluation, but according to Brown (1990), individuals’ implicit beliefs about their ability must 
be taken into account if one is to understand which influence is likely to be acted upon. 

The relationship between achievement goal theory and Brown’s (1990) typology is 
depicted in Table 1: 

Ins ert Table 1 approximately herej 

The differences evident in the low ability condition (cells 1 & 2) are akin to the 
differences one would expect between a performance-avoidant goal orientation and a mastery 
goal orientation respectively. Cell 3 is analogous to the performance-approach goal orientation, 
whereas cell 4 is considered a pure mastery condition. 

Mastery oriented students seek feedback to develop their ability and are likely to remain 
objective in estimating their competence expectations even if such expectations are low. Since 
the diagnostic value of feedback seeking outweighs any potential costs (e.g., ego defense; 
impression management), mastery oriented individuals do not perceive a need to distort 
perceptions of their abilities. In addition, students who adopt learning goals also report greater 
use of cognitive strategies and reports of self-regulation (Middleton & Midgley, 1997; Wolters et 
al., 1996) both of which some researchers believe leads to better self-assessment accuracy. 

Research with high school mathematics students (Martin & Debus, 1998) revealed that a 
task-orientation (“I feel really successful in schoolwork when I solve a problem by working 
hard”) was not predictive of either under- or over-rating. However, only a third of the sample 
(e.g., the medium actual ability) was used in this analysis, and so measures of over-rating were in 
reference to students who held high mathematics self-concepts, and under-rating was indicative 
of low self-concepts. The present study argues that individuals adopting a mastery goal focus 
will hold competence expectations that match their actual performance — that is they will be well 
calibrated. 
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Individuals who adopt performance-avoidance goals are very skeptical of ability 
development, and due to the heavy costs (e g., effort, self-presentation, and ego) associated with 
feedback-seeking processes, are likely to “low ball” their expectations in order to avoid 
demonstrating a further lack of ability. Individuals with a performance-avoid goal orientation, 
therefore, will be miscalibrated toward under-confidence. This logic parallels the claim made by 
Elliot and Church (1997) who indicated that performance-avoidant individuals were rooted in a 
fear of failure and held low expectation of competence. 

The work by Phillips (1984) supports this argument. Children with low perceived 
competence displayed lower achievement standards than those children with average or high 
perceived competence. Children in the study maintained expectancies for success that “seriously 
underestimated the actual level of performance their test scores revealed they could achieve” (p. 
2010). Children “who disparaged their abilities” utilized a strategy that serves to protect their 
already “precarious judgments of competence from internalized sources of criticism and 
feedback” (p. 2010). 

Performance-approach goals are related to high competence expectations (Elliot & 
Church, 1997). Individuals espousing performance-approach goals are likely to be overconfident 
because such a situation provides an opportunity to demonstrate successfully one’s competence. 
If the perception that success is possible, a performance-approach individual will likely “go for 
it.” Support for this claim comes from the research that Martin and Debus (1998) conducted. 
Students with an ego-orientation (“I feel really successful in schoolwork when I know more than 
other people”) held high mathematics self-concepts and were more likely to over-rate their 
perceptions. 

Based on the preceding logic, the relationship between goal orientation and discrepancy 
score (Linkage 2 in Figure 2) will exhibit the following pattern: Mastery (calibrated), 
performance-avoidant (underconfident), and performance-approach (overconfident). 
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Attention now turns to a discussion of linkage 3. Discrepancies with a large negative 
valence occur when one’s expectations exceed one’s actual abilities — that is, a state of 
overconfidence. According to social cognitive theory (Bandura, 1991) , feedback of this type is 
likely to lead to the greatest amount of dissatisfaction for an individual, and heightened 
motivation, whether in terms of increased effort or the likelihood of engaging in remedial 
behaviors such as help-seeking, is likely to occur. 

Research studies on calibration accuracy have concluded that over-rating leads mainly to 
adaptive outcomes. Ilardi, Leone, Kasser, and Ryan (1993) studied discrepancies between 
employee and supervisor ratings of work motivation. These researchers demonstrated that 
workers who over-rated their self-perceptions of autonomy, competence, and relatedness 
compared to the supervisor’s perceptions showed significantly higher levels of general job 
satisfaction and marginally higher levels of specific job task satisfaction than under -raters. 

In a study by Karabenick and Knapp (1991), college students were presented the scenario 
of not performing as well as they wanted. Under this negative discrepancy condition, these 
researchers found that students’ intentions to engage in achievement-related activities were 
related to help-seeking tendencies (e.g., trying harder, obtaining help from instructors, and 
asking classmates for assistance). 

Based on the studies discussed in this section, the following testable research hypothesis 
is presented: Calibration accuracy (as indexed by discrepancy scores) will mediate the 
relationship between students’ goal orientation and their post-assessment motivation (Hypothesis 

3). 

Mediational Route #2 

This discussion focuses on linkages 5 and 6 in Figure 2. The present study investigates 
three proposed cognitive mediators (fairness, utility, and perceived threat). Theory supportive of 
each construct will be reviewed in succinct measure. 

The present study suggests that students adopting mastery goals will hold higher 
perceptions of fairness for the assessment process compared to students who adopt performance 
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goals. According to Dweck (1988), attributes that influence outcomes are perceived as mutable 
and thus perceptions of control are more easily generated from an incremental perspective 
(mastery orientation) compared to an entity perspective (performance orientation). As such, a 
low or negative attribute (e.g., inadequate fairness) has the potential to be altered and desirable 
outcomes achieved from an incremental view compared to an entity view. 

Attribution theory (Weiner, 1986) lends additional support for this claim. In a research 
context, Li den and colleagues (Liden, Ferris, & Dienesch, 1988; Li den & Mitchell, 1985) found 
that study participants who attributed success internally held higher perceptions of appraisal 
fairness and accuracy as well as satisfaction with the feedback session compared to participants 
who attributed success externally. 

Learning goals have been found to be positively and significantly related to the perceived 
value of feedback seeking (VandeWalle, Ganesan, Challagalla, & Brown, 2000), whereas 
performance-avoidance goals have formed negative relationships with perceived value, and 
performance-approach goals have been found to be unrelated (VandeWalle & Cummings, 1997) . 
Value is in reference to how useful individuals find specific feedback. 

VandeWalle et al. (2000) found a significant negative relationship between a learning 
goal orientation and perceived cost (p = -0.22, £ < .001) of feedback-seeking, and VandeWalle 
and Cummings (1997) found significant relationships between perceived cost of feedback- 
seeking and all three goal orientation dimensions: Learning (r = -0.38, g < .001), performance- 
avoid (r = 0.47, £ < .001), and performance-prove (r = 0.18, £ < .01). 

Ryan and Pintrich (1997) found significant and sizeable relationships between goal 
orientation and attitudes toward help-seeking. Task goals were negatively and significantly 
related to appraisals of threat from teachers (p = -0. 1 6, g < .05). Both an extrinsic and relative 
ability goal orientation positively and significantly predicted threat appraisals from teachers (p = 
0.21, g < .01 and p = 0.24, g^< .001 respectively). Based on these studies, it is expected that 
perceived threat will be greatest for performance-avoidant individuals followed, in decreasing 
order, by performance-approach individuals and then mastery-oriented individuals. 
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The three cognitive mediators are expected to form positive significant relationships with 
students’ post-assessment motivation. The construct fairness typically is a consistent, positive 
indicator of goal directed behavior. Gilliland and Langdon (1998) , for example, found a positive 
relationship between perceptions of fairness and employees’ motivation to improve, and 
Quinones’ (1995) study demonstrated that trainees’ fairness perceptions significantly predicted 
trainees motivation to learn (P = .29, p < .01). 

VandeWalle (1995) found a significant positive correlation between the perceived value 
of feedback and feedback-seeking intentions in both the academic (r = .51, £< .001) and work (r 
= .55, p< .001) domains. Ryan and Pintrich (1997) found that students’ attitudes toward help- 
seeking predicted subsequent help-seeking behavior. A focus on the benefits of help-seeking 
positively predicted adaptive help-seeking behaviors (P = 0.3 1, p< .001). Related research has 
documented significant, positive relationships between academic self-efficacy and both intrinsic 
value (Pintrich & De Groot, 1990) and task value (Garcia & Pintrich, 1996). 

VandeWalle (1995) found a significant negative correlation between perceived cost of 
feedback-seeking and feedback-seeking intent in both the academic (r = - . 15, p <.05) and work 
(r = -.42, p < .001) domains. Ryan and Pintrich (1997) found that a focus on threats positively 
predicted avoidance of help seeking regardless if perceptions were aimed toward teachers (P = 

0. 16, p < .05) or peers (P = 0. 19, £< .01). In a study of feedback in a college statistics course, 
Stubblebine (1998) concluded that students who classified feedback as negative and threatening 
lowered their self-goals and also significantly lowered their self-efficacy compared to students 
who did not view the feedback as outcome threatening. 

Based on the research reviewed in this section, the following hypothesis is presented: 
Students’ cognitive perceptions of utility, fairness, and perceived threat will mediate the 
relationship between students’ goal orientation and their post-assessment motivation (Hypothesis 
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Mediational Route #3 

This mediational route adds linkage 4 to the theoretical model (Figure 2). Research on 
assessment centers intimates that post-assessment outcome measures such as participants’ 
attitudes and intentions may be determined directly by the assessment decision (e.g., pass/fail) or 
determined jointly by the decision and assessment procedures used (Robertson et al., 1991). The 
joint influence explanation posits that participant reactions to the assessment process mediate the 
link between assessment results and post-assessment attitudes and intentions. In one sense, 
students’ feedback reports from the assessment serve as the “assessment decision” because it 
answers in students’ mind the question: “How did I do on this assessment?” 

Social learning theory (Bandura & Cervone, 1983) and control theory (Carver & Scheier, 
1990) provide a theoretical justification that can account for this proposed mechanism. Both 
theories contend that individuals’ reactions to feedback discrepancies are mediated by self- 
evaluative mechanisms. Individuals’ satisfaction, goal setting, and subsequent performance are 
affected by the valence (sign) of received feedback. A negative discrepancy indicates that one’s 
progress toward an established behavioral standard is lacking, and heightened effort results from 
both the self-dissatisfaction with substandard performance and the anticipated self-satisfaction 
for matching future accomplishments under the condition that the discrepancy is viewed as 
reducible and one remains self-assured of their abilities 2 . 

The exact nature of this self-evaluative mechanism is most likely context dependent. In a 
developmental setting, for example, Robertson et al. (1991) demonstrated that career stage is a 
contextual factor, which subsequently determines which cognitive mediator is in operation. For 
participants in the latter stages of their careers (i.e., stabilization & establishment) measures of 
“adequacy” (procedural justice effects) fully mediated the relationship between outcome decision 
and the dependent measure (withdrawal cognitions). Contrarily, for participants in early stages of 



2 Discrepancy, according to control theory, is in relation to a sought after goal or standard. In this study, discrepancy 
is in relation to gaps between one’s judgements of one’s competence to perform a task compared to an objective 
measure of the same. 
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their careers (i.e., exploration), the variable “career impact” (not “adequacy”) served as the 
cognitive mediator in predicting participants’ withdrawal cognitions. In contrast, Francis-Smythe 
and Smith (1997) studied a selection context and found that career impact mediated the 
relationship between the outcome decision and participants’ subsequent affective reactions; 
support was not garnered for the role of the adequacy of the process. 

In summary, individuals who over-rate their abilities will report the greatest amount of 
self-dissatisfaction, which, in turn, is expected to affect students’ cognitive perceptions of utility, 
fairness, and perceived threat. The literature reviewed in this section leads to the final research 
hypotheses for this study: Students’ calibration accuracy (as indexed by their discrepancy scores) 
and cognitive perceptions will jointly mediate the relationship between students’ goal orientation 
and their post-assessment motivation (Hypothesis 5). 

Method 



Sample and Design 

Participants in this study included two hundred ten (n = 210) secondary students enrolled 
in vocational and technical education programs from four counties in Southeastern Michigan. 
Students were primarily Caucasian (84%), male (95%) and were distributed mainly in the 1 1 th 
and 12 th grades (90%). Students were enrolled in a variety of programs including machine 
tooling, pre-engineering, drafting, plastics, and robotics. 

Given the exploratory nature of this study, a correlational research design was utilized to 
investigate the relationships between variables (Gall, Borg, & Gall, 1996). The constraints and 
affordances of the research context did not support the possibility of an experimental or quasi- 
experimental design. A comparison or control group was not possible since participants were not 
to be disadvantaged in any way by withholding a potentially valuable intervention. In addition, 
sustained access to interact with assessment participants was not possible, and so qualitative 
design approaches were not utilized. 
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Procedure 

Students attended three sessions, each of which lasted approximately one hour and were 
separated by no more than a week. The first session served as an introduction and orientation to 
the assessment process. Students learned that skill assessment using the AccuVision ™ workplace 
Success Skills (WSS) assessment instrument was an integral component of a statewide aim 
(Michigan Manufacturing Technology Program) to improve vocational and technical education. 

Students were shown a short introductory video meant to motivate them by illustrating 
the importance of interactive skills (e.g., soft skills) in manufacturing sectors. Students were 
engaged in a discussion of the changes that are occurring in the workplace: Limited and defined 
job tasks of the past (i.e., specialization) are being replaced with multiple job tasks requiring 
cross training and job rotation in the present (i.e., generalization). Students practiced answering 
questions of similar content and format to those found in the actual assessment. The first set of 
survey data — which included demographic information, students’ competence expectations, goal 
orientation, and perceived ease of computer use — were collected during this introductoiy 
session. 

During the second session, students registered for and took the AccuVision module 
entitled “Interacting with Others.” Registration occurred “online” using the actual assessment 
software, and students inputted specific demographic information including age, race/ethnicity, 
and job status (e.g., incumbent). 

The assessment module took approximately 65 minutes to complete. Students were 
presented with ten situational videos that depict relevant workplace problems or dilemmas. In 
one situation, for example, an employee fields a complaint over the phone from an irate customer 
who demands that the problem (delivery of bolts with wrong dimensions) be fixed immediately. 
The employee, however, is the only worker in the shop at the time; the supervisor is out of the 
office. The issue facing the employee, then, is how to deal appropriately with the irate customer 
given the circumstances surrounding the situation. 
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There are two types of questions for each video situation. The first type (of which there 
are two questions) asks students to select the two solutions (out of four possible) that represent 
the most effective and least effective means of resolving the given problem. The four possible 
solutions are acted out (video and audio component) and are presented graphically in text form 
with an accompanying narration. The second type of question consists of a single item that is of 
the standard multiple choice variety (i.e., pick the single best answer). This latter type of 
question is more factual in nature, and the possible answers are presented only in text form with 
narration rather than the video simulation format. 

Students uploaded their completed assessment to a scoring server using FTP (File 
Transfer Protocol) via the Internet. The assessments were scored using a separate piece of 
administrative software, and individualized elaborative feedback reports were generated for each 
assessment participant. 

The third session was devoted to feedback mediation. Students received individualized, 
elaborative feedback reports and were guided through the various sections of the report in an 
interactive format. Students learned their skill strengths and weaknesses, received information 
about which skills (if any) they under- or over-estimated their abilities, and learned of 
recommended strategies to improve their skills. Students participated in a discussion that 
facilitated interpretation of the feedback (i.e., “What does it mean if your actual score was 
greater than your predicted score?”). Students were asked of some possible ramifications if they 
approach new work or learning activities being either extremely over -confident or extremely 
under-confident in their abilities relative to the task at hand. 

The session ended with the collection of the final wave of survey data: Student 
perceptions of the fairness, utility, and perceived threat of the assessment process; confidence 
perceptions (eg., self-efficacy) to bridge any skills gaps identified as part of the assessment 
process; willingness to seek additional feedback opportunities (e.g., feedback-seeking intent). 
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Measures 



Assessment Instrument 

The AccuVision Workplace Success Skills (WSS) assessment instrument measures 
participants’ skill levels in five areas: (a) Interacting with others, (b) listening, (c) trainability, (d) 
structuring work activities, and (e) graphs & charts. These skills are in alignment with the 
competencies recommend by The Secretary's Commission on Achieving Necessary Skills 
(SCANS, 1991) and are compatible with those skills identified for high performance entry-level 
manufacturing positions (National Coalition for Advanced Manufacturing [NACFAM], 1997). 
The assessment technology includes job simulation in CD-ROM format that incorporates 
situational videos. This study investigated the first assessment module, “Interacting with Others,” 
which assessed individuals in five sub-skill areas. This study will present results from two of 
these areas: Customer service orientation (“taking actions to meet or exceed the performance 
expectations of others”) and problem solving (“exercising sound judgment and reason in 
determining courses of action to pursue”). The assessment instrument is produced by AlignMark 
Corporation 3 and was distributed by the Michigan Virtual University 4 for the sake of this study. 
Statistical Controls 

Mohr (1995) recommends gathering data on those variables where participants likely 
differ and for which there is theoretical justification for including the variables into the model 
and subsequent analysis. 

Prior research, for example, has demonstrated that low self-efficacy levels can form 
barriers to specific training techniques including the use of computerized multimedia training 



3 http://www.alignmark.com The AlignMark Company is a leading provider of Internet and technology-based 
human resources solutions engineered to assist Fortune 500 companies and other organizations in improving 
productivity. 

4 http ://www,mivu , org The Michigan Virtual University (MVU) is a private, not-for-profit 501(c)3 Michigan 
corporation established in 1998 to meet the specific workforce development, education, and training needs of 
Michigan businesses and industries and their current and prospective employees through the innovative use of 
electronic learning technologies. 
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(Christoph, Schoenfeld, & Tansky, 1998). Since the A ecu Vis ion assessment instrument is 
technology -mediated, attention to participants’ beliefs about (and attitudes toward) computer 
technology use were taken into consideration. The variable “perceived ease of computer use” 
served as a control (proxy) to represent participants’ self-efficacy percepts entering the 
technology-mediated assessment process. 

Five scale items were drawn from Venkatesh and Davis (1996) who found that 
individuals anchor their computer ease of use perceptions on computer self-efficacy before 
hands-on system use. Students responded to statements (e.g., “I find it easy to get a computer to 
do what I want it to do.”) on a 7 -point Likert-type scale ranging from 1 (strongly disagree) to 7 
(strongly agree). The internal consistency of the five items indicates very good reliability (a = 
0.94). 

Prior research has demonstrated differential effects between goal orientation and both 
gender (Bouffard, Boisvert, Vezeau, & Larouche, 1995) and race/ethnicity (Midgley, 

Arunkumar, & Urdan, 1996). Thus, additional statistical controls included measures of these 
variables along with students’ grade level and self-reported grade point average (GPA). 

Goal Orientation 

The goal orientation items were adopted from VandeWalle (1997) and were modified for 
the present study. Items were operationalized at a “mid-level” of specificity to enhance 
predictive validity since it is possible that individuals could hold different goal orientations in 
different domains (e.g., athletic, academic, and work). In alignment with contemporary 
motivational theory (Elliot & Church, 1997; Midgley et al., 1998; Pintrich, 2000b), the construct 
was operationalized as three-factors. 

Students responded with a variant of a Likert-type scale ranging from 1 (Not at all true of 
me) to 7 (Very true of me). For the mastery scale (“I often look for opportunities to develop new 
skills and knowledge”) a = 0.79; For the performance-approach scale (“It is important that I 
show I can perform better than my coworkers”), a = .70; For the performance-avoid scale (“I 
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would avoid taking on a new task if there were a chance that I would appear less able than 
others.”), a = 66. 

A confirmatory factor analysis was performed on the goal orientation items using the 
three-factor model. The analyses were run with the AMOS' statistical software package 
(Arbuckle, 1999). Multiple indices were used to evaluate the fit of the model including the 
Goodness-of-fit index (GFI), the Tucker-Lewis index (TLI), the Comparative fit index (CFI), 
and the Root Mean Square of Error Approximation (RMSEA). Each index evaluates the fit of the 
model slightly differently, and so one can have increased confidence in a model if the various 
indices converge (Hu & Bentler, 1995; Maruyama, 1998). 

For the first three indices, a critical value of .90 is established as a minimum criterion for 
establishing the adequacy of a model’s fit. For the RMSEA index, a value lower than .08 
indicates an adequate fit whereas values less than .05 indicate a good fit (Browne & Cudeck, 
1993). The RMSEA is accompanied by a p value for a significance test with a critical value of 
.05 since the index tests for the approximation of the model fit to the population. A non- 
significant p value, therefore, is indicative of a close fit. 

The fit statistics for the model using actual goal orientation data are as follows: % 2 (df = 
32) = 56.83, p = 0.004, GFI = 0.950, TLI = 0.935, CFI = 0.954, RMSEA = 0.059 with p (0.05) = 
0.252. When compared to prior research and based on the recommendations for determining 
model fit, the goal orientation items in the present study fit the three-factor goal orientation 
model adequately. 

Competence Expectations 

Meece and Courtney (1992) define achievement expectancies as subjective probabilities 
of success on a task. The construct is related conceptually to estimates of self-confidence and 
self-efficacy in achievement contexts. Expectancy perceptions are also related to self-concepts of 
ability but are more predictive of achievement outcomes than are the more global assessments of 
one’s capabilities. In addition, expectancies of success are influenced by factors other than 
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perceived competence or efficacy and therefore cannot be equated with these constructs (Assor 
& Connell, 1992). 

In the present study, participants were presented with a series of task statements that are 
representative of tangible on-the-job behaviors. There were four task statements per sub-skill 
contained in the “Interacting with Others” module. Sample items for the Customer Service 
Orientation skill-set include: “Help co-workers with work problems (for example, getting behind 
schedule)” and “Work with people from other work areas to solve common problems.” Sample 
items for the skill-set Problem Solving include: “Identify which parts of your job are most 
important” and “Identify work methods that are not productive.” Students were presented with a 
series of these task statements and were instructed to reflect and to estimate their ability to 
perform the skill on-the-job. Students recorded their competence expectations for these tasks on 
a scale ranging from 0 (low ability) to 10 (high ability). 

Calibration 

Horgan (1992) defines calibration as the accuracy with which one can predict one’s 
performance. Students in this study were considered “calibrated” if their competence 
expectations for their performance on the assessment correspond closely to their actual 
assessment performance 5 . Miscalibration occurred when students’ perceived judgments diverged 
from their true performance due either to over- or under-confidence (c.f., Schraw, Potenza, & 
Nebelsick-Gullet, 1993). 

Students’ estimates of their competence expectations were averaged over the four task 
statements for each sub-skill. Estimates of coefficient alpha (a) allowed for the investigation of 
the presence of response sets — that is, individuals being overly consistent when judging their 
performances. According to Schraw et al. (1993), moderate degrees of internal consistency (a « 
0.70) indicate that individuals can judge their performance accuracy consistently regardless of 



5 A distribution of discrepancy scores (actual - expected) for each skill-set was determined. Scores within ± 10 from 
the mean were considered calibrated. 
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other constrains such as item difficulty or actual performance, but really high values may 
indicate bias and inaccuracy. Internal consistencies for the two skill-sets in this study were as 
follows: Problem solving (a = 0.72) and customer service orientation (a = 0.75). 

Mean scores for each sub-skill were used to create expectancy tables (i.e., percentile 
scores) which could then be compared to students’ actual scores using a common metric (e.g., 
percentiles). Calibration accuracy is indexed as a discrepancy score such that high values 
represent under-rating of abilities while low values indicate over-rating. This difference variable 
is continuous and is represented algebraically as follows: 

accuracy = actual - predicted 

Utility 

The term utility represents one’s perceptions of the potential value of feedback sought for 
realizing a particular feedback-seeking motive (e.g., diagnostic) (Morrison & Bies, 1991). Such 
operationalization parallels the construct “value” in feedback-seeking contexts such as 
classrooms (Pintrich, Roeser, & De Groot, 1994) and assessment centers (Francis-Smythe & 
Smith, 1997). Students responded to five statements (e.g., “Learning about suggested strategies 
that can improve my performance is valuable information”) on a 7-point Likert-type scale 
ranging from 1 (Strongly disagree) to 7 (Strongly agree). These items showed good internal 
consistency (a = 0.85). 

Fairness 

Research has shown that general perceptions (attitudes) toward testing are positively and 
significantly related to procedural justice perceptions above and beyond the effects due to 
outcome results (Bauer, Dolen, Maertz Jr., & Campion, 1998) . Based on a review by Arvey and 
Sackett (1993), items addressing the fairness construct focused on content, context, process, and 
outcome factors. Items were taken from several sources (Arvey et al., 1990; Bauer et al., 1998a; 
Bauer et al., 1998b; Lounsbury et al., 1989) and were subsequently adopted and modified for the 
present study. 
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Students responded to 16 items on a 7-point Likert-type scale ranging from 1 (Strongly 
Disagree) to 7 (Strongly agree). Three separable constructs emerged from the study. What 
follows are representative items and values for Cronbach’s alpha: Chance to perform (a = 0.83; 
“The feedback report is an accurate evaluation of my performance”); Test difficulty ( a = 0.61; 
“During the test, I often thought about how poorly I was doing”); Job-relatedness (a = 0.62; 

“The test does not really measure a person’s true ability for a beginning-level manufacturing 
job”; reverse scored). 

These items were included in a confirmatory factor analysis using the AMOS statistical 
software package (Arbuckle, 1999). The fit statistics for the resulting model are as follows: % 2 
(df = 24) = 24.65, p = 0.425, GFI = 0.974, TLI = 0.998, CFI = 0.999, RMSEA = 0.01 1 with p 
(0.05) = 0.896. The three factors determined from this analysis are in general agreement with 
what has been presented in the literature (Bauer et al., 1998a; Bauer et al., 1998b) . 

Self-Efficacy 

Bandura (1982) defines perceived self-efficacy as a judgment of “how well one can 
execute courses of action required to deal with prospective situations” (p. 122). Judgments of 
self-efficacy are task specific, and so measures of self-efficacy need to be tailored to the criterial 
task that is being assessed in the domain of functioning (Pajares & Miller, 1995). Also, since 
efficacy beliefs differ in magnitude, generality, and strength, different levels of tasks within the 
domain need to be considered when developing accurate measures of accuracy — doing so 
provides for the greatest explanatory and predictive power (Bandura, 1995) . 

In the present study, the variable was operationalized as participant’s confidence to 
bridge skill deficiencies identified by the assessment process. Two sample items for this 
construct are: “How confident are you that you could complete a class that teaches you how to 
improve your skills,” and “How confident are you that you could learn what it takes to improve 
your skills.” The scale was anchored at three points: 0 (“Cannot do”), 5 (“Moderately certain can 
do”), and 10 (“Certain can do”). The five scale items demonstrated high internal consistency (a 
= 0.89). 
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Feedback-Seeking Intent 

The construct feedback-seeking intent represents an intention category relevant to 
feedback responses (Fedor, 1991). VandeWalle (1995) combined the original work on feedback- 
seeking (Ashford & Cummings, 1983) and Ajzen’s Theory of Planned Behavior (1991) to 
demonstrate empirically the predictive validity of one’s willingness to seek feedback as an 
outcome, or dependent, measure in feedback studies. 

In the present study, students completed items on a 7-point Likert -type scale ranging 
from 1 (Strongly disagree) to 7 (Strongly agree). Representative items include statements such as 
“How willing would you be to meet with a school counselor to learn of other ways of measuring 
your skills” and “How willing would you be to meet with an employer to learn how your skills 
are related to a job.” The six scale items were highly reliable (a = 0.84). 

Analytic Strategy 

Each of the two dependent measures — feedback seeking intent and self-efficacy — were 
regressed onto the endogenous variables. The analyses were conducted in several steps — a 
procedure referred as hierarchical multiple regression. First, the motivational variables (e.g., goal 
orientation) were regressed on the statistical control variables (e.g., gender, race/ethnicity, GPA, 
grade level, and perceived ease of computer use) thereby assuring control for these variables 
during all analyses. In the second step, the discrepancy scores were regressed on the goal 
orientation variables. The third step involved regressing the potential cognitive mediators 
(fairness, utility, and perceived threat) on both the discrepancy scores and the motivational 
variables. The bivariate correlation between perceived threat and utility, however, was 
significantly large (r = .79). Such a high value is likely indicative of multi-collinearity 6 , and so 



6 One index of multi-collinearity is the Variance Inflation Factor (VIF). Values of VIF > 5 are indicative of highly 
problematic collinearity (Belsley, Kuh, & Welsch, 1980). For the construct perceived threat, the VIF > 3, which is a 
cause for concern. 
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perceived threat was omitted from subsequent analyses. The last step involved independently 
regressing each of the outcome measures on the motivational variables, discrepancy scores, and 
perception variables. 

From this analysis, tentative path models illustrating the significant relationships between 
the variables were created. An additional set of regressions were performed for each proposed 
model, and diagnostic statistics revealing influential data points were obtained — specifically 
Cook’s distances and standardized DFFIT values. Threshold cutoff values of .05 and ± .85 for 
these respective statistics were used to detect influential data points. Data exceeding these 
threshold values were considered influential and were omitted from subsequent analyses (Belsley 
et al., 1980, Draper & Smith, 1981). 

The data were then analyzed using techniques of structural equation modeling. Analyses 
were run with the AMOS statistical software program (Arbuckle, 1999) . The use of composites 
(i.e., scales) rather than multiple indicators when estimating parameters in structural equation 
models for studies with small sample sizes has been justified (Liang, Lawrence, Bennett, & 
Whitelaw, 1990). This procedure resulted in path models for each skill-set that illustrates the 
relationships between the variables in this study. Analysis of these path diagrams with structural 
equation modeling allows for inclusion of multiple dependent variables, makes use of the 
covariation between all the variables in the full model, provides indices of the “fit” of the data to 
the proposed structure, and illustrates in clear fashion mediational routes. 

Results 



Descriptive Statistics 

Students’ skill scores (at the percentile level) and mean discrepancy scores for each sub 
skill area are represented in Table 2: 



Insert Table 2 approximately herej 
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On average, students underestimated their abilities most with respect to the problem 
solving skill-set and overestimated their abilities the most in relation to customer service. The 
presentation of results and ensuing discussion will focus on these two cases. Table 3 contains 
Pearson product correlations, means, and standard deviations for the variables in this study. 

jlnsert Table 3 approximately herej 

Hierarchical multiple regression results for the two skill-sets appear in Tables 4 and 5. 
Based on the regression results, fully estimated path models for the two skill-sets were tested and 
appear in Figures 1 and 2: 

jlnsert Figures 3 and 4 approximately here 

The following statistics indicate that the models adequately fit the data. For customer 
service orientation (Figure 3): x 2 (df = 30) = 42.44, p = .07, GFI = 0.961, TLI = 0.896, CFI = 

0.93 1 , RMSEA = 0.045 with P (0.05) = 0.577, and for problem solving (Figure 4): % 2 (df = 38) = 
47.7 1 , p = . 1 34, GFI = 0.960, TLI = 0.944, CFI = 0.962, RMSEA = 0.035 with P (0.05) = 0.775. 
Main (Direct) Effects 

The direct effect between goal orientation and feedback seeking intent (Hypothesis 1) 
was confirmed with minor qualifications. For the skill-set customer service orientation, students 
with high mastery goals indicated, on average, greater levels of feedback seeking intent (P = 

.14), but the result did not achieve statistical significance. When examining the path model 
(Figure 3), however, mastery goals do predict significantly higher levels of feedback seeking 
intentions (P= .17, p < .05). 

As predicted, there was no significant discernible relationship between students’ 
performance-approach goals and feedback seeking intent (P = .06, n.s.). Contrary to prediction, 
however, the inverse relationship between performance-avoidance goals and feedback seeking 
intent was not found (P = -.02, n.s.). 

The direct effect between goal orientation and self-efficacy (Hypothesis 2) was partially 
confirmed. Students adopting high levels of mastery goals tended, on average, to report higher 
levels of self-efficacy for both customer service orientation (P = . 17, p < .05) and problem 




27 



Goal Orientation and Calibration 27 



solving (P = . 19, p < .01). For both skill sets, the relationship between performance-avoidance 
goals and self-efficacy was headed in the predicted negative direction, but the results did not 
reach statistical significance (p ave - -.07, n.s.). Contrary to prediction, performance-approach 
goals were positively and significantly predictive of students’ self-efficacy levels for both skill - 
sets (Pave = .15, p < .05). Path models with structural equation modeling corroborate these results 
(Figures 3 and 4). There were gender differences as well with males indicating higher levels of 
self-efficacy on average than females (P ave = .18, p < .01). 

Indirect Effects 

Guidelines for conducting a mediational analysis are well established (Baron & Kenny, 
1986; Judd & Kenny, 1981). The empirical validation of a hypothesized mediational model 
requires that three conditions be met. First, a direct relationship between the predictor variable 
and the dependent (outcome) measure must be established as a necessary condition for 
mediation. Hypotheses 1 and 2 serve this aim. Second, the first link in the mediational chain is 
established if it can be shown that the predictor variable significantly impacts the proposed 
mediator (i.e., Linkages “2” and “5” in Figure 2). Completion of the mediational chain entails 
demonstrating a significant relationship between the proposed mediator and the outcome 
measure while controlling for the effects of the predictor variable (i.e., Linkages “ 3” and “6” in 
Figure 2). 

The logic of mediation argues that if a substantial diminution of the beta coefficient for 
the direct effect occurs when the proposed mediator is inserted, then support for a mediated 
model is evidenced. The remainder of this section presents results to test the 2 nd and 3 rd 
conditions of Baron and Kenny’s (1986) mediational scheme. 

Hypothesis 3 suggests that calibration accuracy (as indexed by discrepancy scores) will 
mediate the relationship between students’ goal orientation and their post-assessment motivation. 
This hypothesis was confirmed but not for all goal orientations. 
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The regression results indicate that performance-avoidant goals were not predictive of 
participants’ discrepancy scores; the standardized regression coefficient was essentially zero and 
non-significant. 

Both mastery (P = 18, p < .05) and performance- approach (P = -.23, p < .01) goals 

were significant, negative predictors of students’ discrepancy scores for the skill-sets customer 
service orientation. Thus, adoption of higher levels of either goal orientation implies less under- 
estimating by students on average. 

Students’ discrepancy scores significantly and negatively predicted students’ feedback 
seeking intentions for the subskill customer service orientation (P = -. 13, p < .05) suggesting that 
over-rating by students led to significantly higher levels of feedback seeking intentions. 

Students’ discrepancy scores, however, did not significantly predict post-assessment self- 
efficacy levels. 

Hypothesis 4 stated that students’ cognitive perceptions of utility, fairness, and threat 
would mediate the relationship between students’ goal orientation and their post-assessment 
motivation. Overall, this hypothesis was not confirmed. There were instances in which the 
individual difference variable goal orientation (as well as certain statistical control variables) 
significantly predicted students’ cognitive perceptions, but the perception variables failed, in 
turn, to predict significant relationships with the outcome measures. Hence, mediation was not 
established. Conversely, there were instances in which perception variables significantly 
predicted outcome measures, but due to non-significant relationships with the independent 
(predictor) variables, mediation was not established. 

Hypothesis 5 posits that students’ calibration accuracy (as indexed by their discrepancy 
scores) and cognitive perceptions would jointly mediate the relationship between students’ goal 
orientation and their post-assessment motivation. This hypothesis was confirmed (with 
qualifications) and only for mastery goals and the problem solving skill-set. 

The beta coefficient for the relationship between mastery goals and self-efficacy changes 
from .23 (p < .01) to .26 (p < .001) when discrepancy scores are included and to .19 (p < .05) 
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when utility is added thus suggesting partial mediation through this complex channel as well. 
Asher’s (1983) method of decomposing relationships between variables into simple and 
compound paths provides additional insights. The strengths of the simple “indirect effect” via 
utility ( 095) and the more complex mediational channel via discrepancy scores and utility 
together (.016) are very much attenuated in comparison to the “direct effect” between mastery 
goal and self-efficacy (.17). Thus, students’ post-assessment self-efficacy levels are best 
predicted directly by their level of mastery goal adoption. 

Results from the hierarchical regressions indicate that the linkage between mastery goals 
and feedback-seeking intent is significant and does not change when discrepancy scores are 
included (P = .20, p < .05), but drops to 0. 13 and is no longer significant once utility is added. 
The direct linkage from mastery goals to utility not considering discrepancy scores had been 
non-significant (i.e., a “suppressor effect”), when the discrepancy scores were omitted. Thus, the 
proposed path for mediation occurs via the complex channel calibration accuracy then utility. 

It is important to note, however, that this proposed route serves as an “indirect effect” 
from mastery goals to utility. Examination of the path model (Figure 4) and application of the 
methodology suggested by Asher (1983) indicates that the absolute strength of this indirect path 
(.04) is markedly smaller than the “direct effect” between mastery goals and utility (.25). Thus, 
knowledge of overestimation of skills adds little, incrementally, to mastery-oriented students’ 
perceptions of the utility of the assessment process. Nevertheless, it is this knowledge of 
underestimation (of problem solving skills in particular) that helps shape the usefulness of the 
feedback for mastery oriented students. It is important to reiterate that students under -estimated 
their abilities the most with respect to the problem solving skill-set (nearly a 15-percentile 
differential). 

Auxiliary Analyses 

A separate set of analyses were run to investigate an alternative hypothesis to self- 
assessment accuracy — i.e., “anchoring effects” (Block & Harper, 1991; Cervone, 1993). A series 
of one-way analysis of variance (ANOVA) were run using students’ competence expectations as 
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the dependent measure and their actual scores (split into three categories of low, medium, and 
high) as a factor. The results were non-significant indicating that regardless of students’ actual 
abilities, there were no significant differences between reported levels of competence 
expectations. 

A one-way analysis of variance (ANOVA) was run using students’ competence 
expectations as the dependent measure and students’ goal orientation levels (i.e., low, medium, 
and high) for each of the three goal orientation constructs (mastery, performance-approach, and 
performance-avoid) as factors. 

For mastery goals, students’ competence expectations increased as the level of goal 
adoption increased, F (2, 207) = 22.97, p < .001 . Similar results occurred for performance- 
approach goals. However, there were no significant differences across performance-avoidance 
goals — a result that runs contrary to extant research by Elliot and Church (1997). 

A similar procedure was ran with students’ actual scores as the dependent measure. 
Results indicate that students’ actual scores did not differ based on the level of either mastery or 
performance (approach or avoidance) goal adoption. 

This sidebar analysis is very revealing. In short, the relationship between students’ 
discrepancy scores and either mastery or performance-approach goal orientation appears to be 
determined by the level of students’ competence expectations. For performance- avoidant goals, 
the level of goal adoption does not bear a relationship with either students’ actual scores or their 
competence expectations. Taken as a whole, these results lend credence to the possibility that an 
anchoring effect may have been in place and will be discussed toward the end of the next section. 

Discussion 

This study sought to explain students’ reactions to skill assessment feedback. Similar to 
the study by VandeWalle and Cummings (1997), the goal orientation construct was placed 
antecedent in the model — as an individual difference variable — to predict students’ post- 
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assessment motivation. The results of the present study allow for an understanding of specific 
mediating variables including the role of students’ calibration accuracy. Results will be discussed 
in relation to potential contributions to education and training research and praxis. 

The first research hypothesis investigated the direct relationship between students’ goal 
orientation and their feedback seeking intentions. The positive relationship found for mastery 
goals and feedback seeking is best understood when reviewing the motives that underlie 
adoption of learning goals — that is, developing the self by acquiring new skills, mastering new 
situations, and improving one’s competence. Farr, Hofmann, and Ringenbach (1993) have 
suggested that employees with a mastery goal focus are perhaps more likely to be proactive in 
seeking out opportunities for learning including self-development and technical training. Results 
from the present study lend empirical credence to this claim. 

Interestingly, performance-avoidance goals and feedback seeking intent were not 
inversely related as was expected. This result contradicts what has been found in the literature 
(VandeWalle, 1997; VandeWalle & Cummings, 1997). One could speculate that in the present 
study, performance-avoidant students did not feel threatened by the proposition that additional 
feedback opportunities might potentially reveal instances of their low ability. Support for this 
claim may stem from the highly developmental nature of the pilot context and because the 
feedback was not provided publicly or mediated by teachers. 

The second research hypothesis investigated the direct relationship between students’ 
goal orientation and their post-assessment self-efficacy. Consistent with the extant literature 
(Middleton & Midgley, 1997), mastery goals were related to high levels of self-efficacy lending 
further evidence to their propaedeutic value for learning. Contrary to prediction, yet consistent 
with some researchers’ findings (Wolters et al., 1996), performance-approach goals positively 
predicted students’ self-efficacy levels. 

Pintrich (2000c) has suggested these “mixed results” are attributable to classroom 
context, which may moderate the relationship. Otherwise, it is quite plausible that students 
focused on approach forms of regulation would have higher personal efficacy beliefs as long as 
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they are successful in “besting others” by demonstrating their high ability. In the present study, 
then, students adopting either mastery or performance approach goals were able to receive and to 
interpret performance feedback and hold high levels of self-efficacy for post-assessment, goal- 
related activities. For approach goals, the developmental nature of the assessment context may 
have contributed to this positive relationship. 

The deleterious effect of adopting avoidance goals on self-efficacy as reported in the 
literature (Skaalvik, 1997) was not detected, although the bivariate correlation was in the 
predicted negative direction ( r = -. 10 ,p- . 14). A large “ceiling effect” was in operation 
whereby high self-efficacy levels (mean 10 out of 10) were evident across all levels (low, 
medium, and high) of avoidance goal adoption. Since research has convincingly shown that 
students oriented toward avoidance forms of regulation exhibit maladaptive patterns of 
cognition, motivation, affect, and behavior (Pintrich, 2000c) , this result must be interpreted with 
caution. 

Research has shown that for highly self-efficacious individuals, repeated negative 
performance feedback leads to a decreased acceptance of the feedback (Nease, Mudgett, & 
Quinones, 1999). Resiliency in one’s confidence despite evidence to the contrary can be quite 
damaging to individuals in that calibration of beliefs to actual abilities is sidestepped. 

This study investigated several potential mediational mechanisms for the relationship 
between students ’ goal orientation and their post-assessment motivation. In the “Interacting with 
Others” module, students over-estimated their abilities the most in relation to the skill-set 
customer service orientation and under-estimated their abilities the most with respect to problem 
solving. Different mediational models resulted depending on whether over- or under -confidence 
was in operation. 

For problem solving, students’ calibration accuracy (as indexed by their discrepancy 
scores) and the perception variable utility jointly mediated the relationship between mastery 
goals and post-assessment motivation. The mediational route established in the path diagram 
suggests that mastery oriented students find the feedback useful, and part of these utility 
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perceptions are determined by their reactions to the knowledge of their level of miscalibration 
(e g., under-estimating). 

This finding is fairly antithetical to what both control theory and social learning theory 
predict (Bandura & Cervone, 1983; Carver & Scheier, 2000). Typically, increased motivation 
stems from feedback that illustrates that one’s performance is below an accepted standard — that 
is, a negative discrepancy. The above findings, however, indicate that positive discrepancies in 
which one’s performance exceeds one’s expectations lead to the positive perceptions of feedback 
utility. Therefore, contrary to established research findings, this study demonstrates one instance, 
or case, where a positive discrepancy leads to heightened motivation. 

Mastery oriented students — with the express goal of skill development and 
improvement — would likely be less satisfied with knowledge of the under-utilization of their 
skill abilities compared to peers adopting other goal orientations. Even though the feedback is 
positive (“My problem solving skills are better than I thought”), a mastery oriented student may 
reflect on potential “losses” such as missed opportunities for skill acquisition, skill development, 
or skill transfer. For mastery oriented students, then, acquiring feedback that specifies under- 
utilization of abilities may serve a motivational function alerting students to excellent 
opportunities for skill improvement (cf., Johnson, Perlow, & Pieper, 1993). Butler’s (1993) study 
is suggestive of such a claim: Individuals oriented toward task goals requested more information 
to clarify task demands and appropriate strategies for the experimental task they were working 
on. 

For customer service orientation, “calibration accuracy” (as indexed by discrepancy 
scores) partially mediated the relationship between goal orientation (the mastery and 
performance-approach dimensions) and students’ feedback seeking intentions. The mediational 
path with the outcome measure self-efficacy did not gamer support. 

Students under-rated their abilities less when adopting higher levels of either mastery or 
performance-approach goals. This study had argued that students oriented toward mastery goals 
would be calibrated accurately while students adopting performance approach goals would tend 
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to over-estimate their abilities, and so the results run counter to the hypothesized claims. Under- 
rating tended to occur with high actual performance while over-rating tended to occur with low 
actual performance — a result that may be due to the presence of an anchoring effect as suggested 
by the auxiliary analyses. 

Feedback seeking was associated with a tendency to over-estimate. This result is credible 
when keeping in mind that over-estimating occurs when students ’ expectations exceed their 
actual performance (negative discrepancy). Seeking additional feedback can provide students 
with the opportunities to bring their expectations more in line with their abilities and possibly 
lead to increases in performance. This result is in line with research that has demonstrated the 
motivating effects of negative discrepancies (Podsakoff & Farh, 1989). 

Students’ discrepancy scores did not significantly predict self-efficacy levels. At first 
glance, this result seems contradictory given the research that has clearly demonstrated a link, or 
mechanism, by which feedback discrepancies are resolved via mediation with one’s level of self- 
efficacy (Bandura & Cervone, 1983). However, the goal orientation construct represents an 
individual difference variable that is placed antecedent in the conceptual framework. Thus, once 
the effects of goal orientation are removed from the discrepancy variable, it is likely that there is 
not enough unexplained variance to account for students’ self-efficacy levels. Examination of the 
path models supports this argument since a direct link between discrepancy scores and self- 
efficacy levels is not evident. In this study, self-efficacy is best explained as a direct effect 
between both mastery and performance-approach goals. These collective results support the idea 
that goal orientation can serve as both feedforward and feedback control (Bandura & Wood, 
1989). 

Limitations 

A limitation of the present study is the fact that longitudinal data were not available since 
ongoing interaction with participants was not possible. Both the magnitude and direction of 
students’ intentions can change over time due to information that was not available (or was 
perceived differently) at the time of initial intention formation (Fedor, 1991). Thus, students do 
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formulate longer-term responses to feedback. Assessment center researchers (in particular, 
Fletcher, 1991; Slivinski, McDonald, & Bourgeois, 1979) urge implementation of studies in 
which longer term effects can be investigated. 

Butler and Winne (1995) claim that the temporal flow of self- regulation cannot 
adequately be studied when focusing on large-scale activities in which aggregate measures are 
drawn over multiple self-regulating incidents as is the case in past self-regulation/feedback 
research. In short, such data fail to reflect “. . .the variance in behavior that is regulation” (p. 246). 
The issue of “grain size” thus represents a second study limitation as multiple and discrete self- 
regulating events could not be assessed. 

Generalizability 

It is tempting to claim that the situational and contextual cues the worker-student 
encounters in a classroom or training center are identical to those found on-the -job. However, 
one must be careful when interpreting the effects of feedback in achievement settings versus on- 
the-job settings because the types of performances, the reasons for the performances, the 
contextual/situational influences that affect the performances, and the tasks/activities that define 
the performances can be quite different. In addition, the potential costs/gains as well as the 
interpersonal context may differ across different settings. The most prevalent factors, which can 
potentially constrain the generalizability of the present research findings to other contexts 
include the participant base, the delivery of the assessment content, and the cues provided during 
feedback mediation. Discussion will center of the first factor. 

The participant base for the present study consisted mainly of male Caucasian high 
school students. Thus, results do not provide an in depth or robust understanding of how females 
or students of a different race/ethnicity would react to similar skill assessment feedback. Given 
the burgeoning amount of research in the area of motivation, self- regulation, and cognition, 
future research needs to be attuned to potential differences across various demographic groups 
(Urdan, 1997). 
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The high school students in this pilot are enrolled in vocational and technical education 
programs and as such are much more likely to enter the world-of-work upon graduation than 
they are to enroll in post-secondary educational programs. The most noticeable exception 
includes students enrolled in school-to-work joint-apprenticeship programs. Even these students, 
however, are not typically enrolled in traditional college preparatory curriculum. The findings 
from this study, therefore, are generalizable to a small cross-section of high school students. A 
study with an adult population or with incumbent workers is recommended. 

Implications 

There are numerous potential contributions to both educational and organizational 
research and practice that this study can provide. The field of human resource and development 
(HRD) relies on feedback that provides developmental recommendations of high diagnostic 
value to enhance individual motivation (Squires & Adler, 1998) as well as broader organizational 
objectives (Francis-Smythe & Smith, 1997). In addition, identifying developmental training 
needs has taken on more import given the use of teams, self-management, telecommuting, and 
less supervisor-subordinate contact (Atwater, 1998). 

Results from this study indicate that students’ perceptions of the utility of the assessment 
process including feedback provision are the most significant predictor of their post-assessment 
motivation. From an applied, practical sense, individuals in charge of overseeing training 
programs need to capitalize on this research finding when the aim is development and 
improvement. To ensure student success in developing their skills and presumably for providing 
opportunities for effective job performance and successful skill transfer, the student-worker 
needs scaffolding leading up to and following both assessment and training programs. Educators 
and employers, for example, can assist the student-worker in reaching their individuals goals by 
providing training and education to repair skill deficiencies in identified areas and receive work- 
related responsibilities that allow them to capitalize on their skill strengths. 

Since research (Colquitt et al., 2000) has demonstrated consistent and strong relationships 
between trainees’ pre-training self-efficacy and their subsequent motivation to learn {r c = .42) 
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and to transfer ( r c = .47), it behooves researchers and practitioners to examine variables (like 
utility) which can be leveraged to influence individuals’ motivation positively. As Zalensy and 
Ford (1990) have suggested, the depth of processing employees devote to different sources (or 
elements) of feedback will likely be an important determinant of the areas in which employees 
will set goals and attempt to change their behavior. 

Directions for future research 

This dissertation is unique from past self-assessment accuracy research due to the 
inclusion of the goal orientation construct — a variable known to significantly covary with 
students’ competence expectations (Elliot & Church, 1997). This approach to self-assessment 
accuracy is different from methodologies used in prior research (e.g., residualized scores, 
Schommer, Crouse, & Rhodes, 1992) the validity of which some researchers have called into 
question (e.g., Assor, Tzelgov, Thein, Ilardi, & Connell, 1990) . 

This study hypothesized that the goal orientation construct would provide results similar 
to what social comparison theory would predict for calibration accuracy — specifically calibration 
for mastery-oriented students, over-confidence for performance-approach goals, and under- 
confidence for students pursuing avoidance goals. These results did not materialize. 

The use of the motivational construct goal orientation — rather than social comparison 
theory — does provide researchers a greater nomological network of variables to study (i.e., self- 
regulation, metacognition, and motivation). Use of such constructs can provide researchers a 
better understanding of feedback related processes compared to social comparison variables 
alone. For example, demonstrating a relationship between input variables (goal orientation) and 
output variables (motivation) in relation to students’ reactions to feedback is an advantage. It 
remains to be seen whether the goal orientation construct can prove worthwhile as an individual 
difference variable for studying the specific feedback effect of calibration accuracy. 
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Figure 1. Conceptual and theoretical model. 
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Figure 2. Abbreviated conceptual and theoretical model. 
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Figure 3, Path model for the customer service orientation skill-set. 




Figure 4. Path model for the problem solving skill-set. 
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Table 1 

Comparison between Goal Theory and Social Comparison Theory in Relation to Perceived 



Self-Enhancement Self-Evaluation 



Note : In Brown’s (1990) study, ability is operationalized as the individual’s perceptions whether feedback is 
likely to reveal that their ability is either low or high. 



(“Performance”) (“Mastery”) 



1. Predicts little information 
seeking because one does not 
want to confront their 
weaknesses 


2. Still predicts information seeking 
despite low ability since benefits of 
obtaining accurate information 
outweigh any ego costs 


3. Predicts information-seeking 
behavior in order to gain 
favorable judgments of one’s 
competence 


4. Predicts information-seeking behavior 
in order to improve one’s competence 



Ability 



£ 

o 



< 



ou 



Table 2 

Mean Percentile and Discrepancy Scores by Sub-Skill 





Percentile Score 


Discrepancy Scores 
(Actual - Predicted) 


Sub Skill 


M 


SD 


M 


SD 


Facilitation 


55.8 


29.2 


+8.1 


39.0 


Influence 


61.6 


29.6 


+11.7 


40.0 


Commitment to Quality 


58.7 


27.1 


+8.1 


41.1 


Customer Service Orientation 


38.3 


28.1 


-13.6 


38.9 


Problem Solving 


64.5 


29.5 


+14.4 


40.7 



Note. Standard deviations are in parentheses. Positive signs (+) denote under-estimating while negative signs (-) 
denote over-estimating. 
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Table 4 

Hierarchical Regression Results (Customer Service) 



Predictor 


p* 


p» 


p c 


R 2 


AR 2 




Feedback Seeking Intent 








Step 0 








.07 




Gender 1 


.00 










Race' 


-.17 










Race f 


-.09 










Grade Level 


-.03 










GPA 


.14* 










Computer 


-.04 










Step 1 








.12 


.05* 


Mastery 


.20* 


.18’ 


.14 






Approach 


.07 


.04 


-.02 






Avoid 


.07 


.06 


.06 






Step 2 








.14 


.02 


Discrepancy Score 




-.12 


-.13* 






Step 3 








.37 


.23*** 


Test Difficulty 






.11 






Job Relatedness 






.07 






Chance to Perform 






.02 






Utility 






.45**’ 






Step 0 


Self-Efficacy 




.03 




Gender d 


.18” 










Race' 


.01 










Race f 


-.05 










Grade Level 


-.11 










GPA 


.01 










Computer 


-.07 










Step 1 








.16 


.13*** 


Mastery 


.23” 


.23** 


.17’ 






Approach 


.17* 


.17* 


.15* 






Avoid 


-.10 


-.10 


-.08 






Step 2 








.16 


.00 


Discrepancy Score 




.00 


-.01 






Step 3 








.31 


.15*” 


Test Difficulty 






.01 






Job Relatedness 






.11 






Chance to Perform 






-.07 






Utility 






.38*” 







Note, p < .05; p<.01.; _p < .001 

‘Standardized regression coefficients without any potential mediating variables. 

Standardized regression coefficients with Discrepancy Score as potential mediator. 

Standardized regression coefficients with Discrepancy Score, fairness measures, and utility added as potential mediators. 
d (Coded Female = 0; Male = 1); '(Coded 1 = Caucasian; 0 = other); f (Coded 1 = minority; 0 = other). 




best copy available 
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Table 5 



Hierarchical Regression Results (Problem Solving) 



Predictor 


p* 


p b 


p c 


R 2 


AR 2 




Feedback Seeking Intent 








Step 0 








.07 




Gender 1 


.00 










Race* 


-.15 










Race f 


-.06 










Grade Level 


-.03 










GPA 


.13* 










Computer 


-.04 










Step 1 








.12 


.05* 


Mastery 


.20* 


.20* 


.13 






Approach 


.07 


.07 


.01 






Avoid 


.07 


.07 


.05 






Step 2 








• 12 


.00 


Discrepancy Score 




-.01 


-.11 






Step 3 








.36 


.24*’’ 


Test Difficulty 






.11 






Job Relatedness 






.09 






Chance to Perform 






.01 






Utility 






45’*’ 








Self-Efficacy 








Step 0 








.03 




Gender d 


.18** 










Race* 


-.01 










Race f 


-.06 










Grade Level 


-.11 










GPA 


.02 










Computer 


-.07 










Step 1 








.16 


.13*** 


Mastery 


.23“ 


.26*** 


.19* 






Approach 


.17* 


.18* 


.15* 






Avoid 


-.10 


-.08 


-.07 






Step 2 








.17 


.02* 


Discrepancy Score 




.14* 


.05 






Step 3 








.31 


.14*** 


Test Difficulty 






.02 






Job Relatedness 






.10 






Chance to Perform 






-.07 






Utility 






.37** 







Note. 2 <.05; p < .01.; _p < .001 

Standardized regression coefficients without any potential mediating variables. 

Standardized regression coefficients with Discrepancy Score as potential mediator. 

Standardized regression coefficients with Discrepancy Score, fairness measures, and utility added as potential mediators. 
d (Coded Female = 0; Male = 1); “(Coded 1 = Caucasian; 0 = other); f (Coded 1 = minority; 0 = other). 




If best copy available 
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